Virtual reality and mixed reality enhanced exercise machine

ABSTRACT

A system for a virtual and mixed-reality enhanced exercise machine that allows a user to exercise while interacting with a virtual reality or mixed reality environment using whole body movements and positioning, and receiving haptic feedback associated with the user&#39;s interaction with the virtual reality or mixed reality environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

This present application is a continuation-in-part of Ser. No.16/011,394, titled “SYSTEM AND METHOD FOR A MIXED OR VIRTUALREALITY-ENHANCED STATIONARY EXERCISE BICYCLE”, and filed on Jun. 18,2018, which is a continuation-in-part of Ser. No. 15/853,746, titled“VARIABLE-RESISTANCE EXERCISE MACHINE WITH WIRELESS COMMUNICATION FORSMART DEVICE CONTROL AND INTERACTIVE SOFTWARE APPLICATIONS”, and filedon Dec. 23, 2017, which is a continuation of U.S. patent applicationSer. No. 15/219,115, titled “VARIABLE-RESISTANCE EXERCISE MACHINE WITHWIRELESS COMMUNICATION FOR SMART DEVICE CONTROL AND VIRTUAL REALITYAPPLICATIONS”, and filed on Jul. 25, 2016, now issued as U.S. Pat. No.9,849,333 on Dec. 26, 2017, which claims the benefit of, and priorityto, U.S. provisional patent application Ser. No. 62/330,642, titled“VARIABLE-RESISTANCE EXERCISE MACHINE WITH WIRELESS COMMUNICATION FORSMART DEVICE CONTROL AND VIRTUAL REALITY APPLICATIONS” and filed on May2, 2016, and is also a continuation-in-part of Ser. No. 15/193,112,titled “NATURAL BODY INTERACTION FOR MIXED OR VIRTUAL REALITYAPPLICATIONS”, and filed on Jun. 27, 2016, which claims the benefit of,and priority to, U.S. provisional patent application Ser. No.62/330,602, titled “NATURAL BODY INTERACTION FOR MIXED OR VIRTUALREALITY APPLICATIONS” and filed on May 2, 2016, and is also acontinuation-in-part of U.S. patent application Ser. No. 15/187,787,titled “MULTIPLE ELECTRONIC CONTROL AND TRACKING DEVICES FORMIXED-REALITY INTERACTION”, and filed on Jun. 21, 2016, which is acontinuation-in-part of U.S. patent application Ser. No. 15/175,043,titled “APPARATUS FOR NATURAL TORSO TRACKING AND FEEDBACK FOR ELECTRONICINTERACTION” and filed on Jun. 7, 2016, now issued as U.S. Pat. No.9,766,696 on Sep. 19, 2017, which claims the benefit of, and priorityto, U.S. provisional patent application Ser. No. 62/310,568, titled“APPARATUS FOR NATURAL TORSO TRACKING AND FEEDBACK FOR ELECTRONICINTERACTION” and filed on Mar. 18, 2016, the entire specification ofeach of which is incorporated herein by reference in its entirety.

This present application is a continuation-in-part of Ser. No.16/011,394, titled “SYSTEM AND METHOD FOR A MIXED OR VIRTUALREALITY-ENHANCED STATIONARY EXERCISE BICYCLE”, and filed on Jun. 18,2018, which is a continuation-in-part of Ser. No. 15/853,746, titled“VARIABLE-RESISTANCE EXERCISE MACHINE WITH WIRELESS COMMUNICATION FORSMART DEVICE CONTROL AND INTERACTIVE SOFTWARE APPLICATIONS”, and filedon Dec. 23, 2017, which is a continuation of U.S. patent applicationSer. No. 15/219,115, titled “VARIABLE-RESISTANCE EXERCISE MACHINE WITHWIRELESS COMMUNICATION FOR SMART DEVICE CONTROL AND VIRTUAL REALITYAPPLICATIONS”, and filed on Jul. 25, 2016, now issued as U.S. Pat. No.9,849,333 on Dec. 26, 2017, which is a continuation-in-part of Ser. No.15/193,112, titled “NATURAL BODY INTERACTION FOR MIXED OR VIRTUALREALITY APPLICATIONS”, and filed on Jun. 27, 2016, which is acontinuation-in-part of Ser. No. 15/187,787, titled “MULTIPLE ELECTRONICCONTROL AND TRACKING DEVICES FOR MIXED-REALITY INTERACTION”, and filedon Jun. 21, 2016, which is a continuation-in-part of U.S. patentapplication Ser. No. 14/846,966, titled “MULTIPLE ELECTRONIC CONTROLDEVICES” and filed on Sep. 7, 2015, now issued as U.S. Pat. No.10,080,958 on Sep. 25, 2018, and is also a continuation-in-part of U.S.patent application Ser. No. 14/012,879, titled “Mobile and AdaptableFitness System” and filed on Aug. 28, 2013, which claims the benefit of,and priority to, U.S. provisional patent application Ser. No.61/696,068, titled “Mobile and Adaptable Fitness System” and filed onAug. 31, 2012, the entire specification of each of which is incorporatedherein by reference in its entirety.

BACKGROUND OF THE INVENTION Field of the Art

The disclosure relates to the field of exercise equipment, and moreparticularly to the field of exercise machines for fitness and health.

Discussion of the State of the Art

Users often interact with fitness devices such as treadmills orelliptical trainers while using a smart device such as their smartphoneor tablet, generally for consuming media. Generally, the interactionswith the fitness device and smart device are separate, with mediaplayback occurring independently of the user's operation of an exercisemachine. For example, while running on a treadmill a user may choose toview a static screen displaying a television show. The elements in atelevision show are static, in that the elements do not change behaviorbased on the user's interactions with the television show, but insteadperform predetermined actions. An example of a dynamic screen is that ofa video game. The user interacts with a remote and influences theactivity of the elements in the video game. At most, the user mayinteract with the screen performing an activity independent of actionsrelated to interacting with the fitness device.

Furthermore, with the rapidly-expanding virtual reality industry, newinteraction methods are being explored including a variety ofcontrollers for gaming, wands, and motion-based input devices includinggloves and camera-based hand tracking. However, these devices all focuson interacting with a user's hands and head movements, and ignore otherparts of the body that could be used to improve interaction andimmersion particularly while a user is within a known definedenvironment such as an exercise machine, while also expanding thepossibilities for data collection.

What is needed, is an exercise machine that is designed to connect to avariety of user smart devices for interaction, that receives directionfrom a connected smart device and provides interaction input to aconnected smart device, and that provides for the use of the human bodyas an input method through position and movement tracking.

SUMMARY OF THE INVENTION

Accordingly, the inventor has conceived and reduced to practice, avirtual and mixed-reality enhanced exercise machine that allows a userto exercise while interacting with a virtual reality or mixed realityenvironment using whole body movements and positioning, and receivinghaptic feedback associated with the user's interaction with the virtualreality or mixed reality environment.

According to a preferred embodiment, the system comprises: an exercisemachine; a mechanical torso joystick comprising: a plurality of rigidarms affixed to the exercise machine; a torso harness affixed to atleast one of the plurality of rigid arms and configured to be worn abouta portion of a user's torso; a plurality of body movement sensors eachaffixed to at least one of the rigid arms and configured to detectmovement of at least a portion of a user's body while the torso harnessis worn during use of the exercise machine; a plurality of feedbackdevices each affixed to at least one of the rigid arms and configured todetect movement of at least a portion of a user's body while the torsoharness is worn during use of the exercise machine; a composition servercomprising at least a plurality of programming instructions stored in amemory and operating on a processor of a network-connected computingdevice and configured to: collect input data based on motion of a user'sbody from the plurality of body movement sensors; produce a virtualjoystick input device that emulates the capabilities of a hardwarejoystick; generate virtual joystick inputs based on the received inputdata; transmit the virtual joystick inputs to a software application;receive feedback from the software application; and direct the operationof at least a portion of the plurality of feedback devices based on thereceived feedback.

According to a preferred method, the method comprises the steps of:measuring, through a plurality of body movement sensors each affixed toat least one of a plurality of rigid arms affixed to an exercisemachine, movement of a user while wearing a torso harness during use ofthe exercise machine; mapping, using a composition server, the measuredmovements to inputs from a virtual joystick device; transmitting thevirtual joystick inputs to a software application; receiving feedbackfrom the software application; and restricting user movements, using aplurality of feedback devices, based on the received feedback, isdisclosed.

According to an aspect of an embodiment, the entire exercise machine ismounted on a motorized, rotatable platform, which rotates the entireexercise machine in the direction indicated by the motion of all or someportion of the user's body.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawings illustrate several embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention according to the embodiments. It will beappreciated by one skilled in the art that the particular embodimentsillustrated in the drawings are merely exemplary, and are not to beconsidered as limiting of the scope of the invention or the claimsherein in any way.

FIG. 1 is a side view of an exemplary variable-resistance exercisemachine with wireless communication for smart device control andinteractive software applications, according to a preferred embodimentof the invention.

FIG. 2 is a top-down view of an exemplary variable-resistance exercisemachine with wireless communication for smart device control andinteractive software applications, according to a preferred embodimentof the invention.

FIG. 3 is a diagram illustrating an exemplary system for a virtualreality or mixed reality enhanced exercise machine, illustrating the useof a plurality of connected smart devices and tethers, and showinginteraction via the user's body as a control stick.

FIG. 4 is a diagram of an exemplary apparatus for natural torso trackingand feedback for electronic interaction, illustrating the use ofmultiple tethers and a movable torso harness.

FIG. 5 is a diagram illustrating a variety of alternate tetherarrangements.

FIG. 6 is a diagram of an additional exemplary apparatus for naturaltorso tracking and feedback for electronic interaction, illustrating theuse of angle sensors to detect angled movement of tethers.

FIG. 7 is a diagram illustrating an exemplary apparatus for naturaltorso tracking and feedback for electronic interaction, illustrating theuse of multiple tethers and a movable torso harness comprising aplurality of angle sensors positioned within the movable torso harness.

FIG. 8 is a block diagram of an exemplary system architecture fornatural body interaction for mixed or virtual reality applications.

FIG. 9 is a block diagram of an exemplary system architecture for astationary exercise bicycle being connected over local connections to asmartphone, an output device other than a phone, and a server over anetwork, according to an aspect.

FIG. 10 is a diagram of an exemplary hardware arrangement of a smartphone or computing device running a user identification component andcommunicating over a network, according to an aspect.

FIG. 11 is a block diagram of a method of mixed or virtual realitysoftware operating to receive input through different sources, and sendoutput to devices, according to an aspect.

FIG. 12 is a diagram illustrating an exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a pluralityof optical sensors to detect body movement of a user during use of anexercise machine.

FIG. 13 is a block diagram illustrating an exemplary hardwarearchitecture of a computing device.

FIG. 14 is a block diagram illustrating an exemplary logicalarchitecture for a client device.

FIG. 15 is a block diagram showing an exemplary architecturalarrangement of clients, servers, and external services.

FIG. 16 is another block diagram illustrating an exemplary hardwarearchitecture of a computing device.

FIG. 17 is a block diagram of an exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a stationarybicycle with hand controls on the handles, and a belt-like harnessattachment.

FIG. 18 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a treadmillexercise machine with a vest-type harness with a plurality of pistons toprovide a hardware-based torso joystick with full-body tracking.

FIG. 19 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a stationarybicycle with a vest-type harness with a plurality of strain sensors andtethers.

FIG. 20 is a flow diagram illustrating an exemplary method for operatinga virtual and mixed-reality enhanced exercise machine.

FIG. 21 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a rotatingplatform, a waist belt and joints providing full range of motion.

FIG. 22 is a diagram illustrating a top view of a waist belt withstraps, buckle, and ball joint providing full range of motion.

FIG. 23 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a movableexercise machine with a rotating platform.

FIG. 24 is a diagram illustrating a top view of pivoting foot steps.

DETAILED DESCRIPTION

The inventor has conceived, and reduced to practice, a virtual andmixed-reality enhanced exercise machine that allows a user to exercisewhile interacting with a virtual reality or mixed reality environmentusing whole body movements and positioning, and receiving hapticfeedback associated with the user's interaction with the virtual realityor mixed reality environment.

One or more different inventions may be described in the presentapplication. Further, for one or more of the inventions describedherein, numerous alternative embodiments may be described; it should beappreciated that these are presented for illustrative purposes only andare not limiting of the inventions contained herein or the claimspresented herein in any way. One or more of the inventions may be widelyapplicable to numerous embodiments, as may be readily apparent from thedisclosure. In general, embodiments are described in sufficient detailto enable those skilled in the art to practice one or more of theinventions, and it should be appreciated that other embodiments may beutilized and that structural, logical, software, electrical and otherchanges may be made without departing from the scope of the particularinventions. Accordingly, one skilled in the art will recognize that oneor more of the inventions may be practiced with various modificationsand alterations. Particular features of one or more of the inventionsdescribed herein may be described with reference to one or moreparticular embodiments or figures that form a part of the presentdisclosure, and in which are shown, by way of illustration, specificembodiments of one or more of the inventions. It should be appreciated,however, that such features are not limited to usage in the one or moreparticular embodiments or figures with reference to which they aredescribed. The present disclosure is neither a literal description ofall embodiments of one or more of the inventions nor a listing offeatures of one or more of the inventions that must be present in allembodiments.

Headings of sections provided in this patent application and the titleof this patent application are for convenience only, and are not to betaken as limiting the disclosure in any way.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or morecommunication means or intermediaries, logical or physical.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required. Tothe contrary, a variety of optional components may be described toillustrate a wide variety of possible embodiments of one or more of theinventions and in order to more fully illustrate one or more aspects ofthe inventions. Similarly, although process steps, method steps,algorithms or the like may be described in a sequential order, suchprocesses, methods and algorithms may generally be configured to work inalternate orders, unless specifically stated to the contrary. In otherwords, any sequence or order of steps that may be described in thispatent application does not, in and of itself, indicate a requirementthat the steps be performed in that order. The steps of describedprocesses may be performed in any order practical. Further, some stepsmay be performed simultaneously despite being described or implied asoccurring non-simultaneously (e.g., because one step is described afterthe other step). Moreover, the illustration of a process by itsdepiction in a drawing does not imply that the illustrated process isexclusive of other variations and modifications thereto, does not implythat the illustrated process or any of its steps are necessary to one ormore of the invention(s), and does not imply that the illustratedprocess is preferred. Also, steps are generally described once perembodiment, but this does not mean they must occur once, or that theymay only occur once each time a process, method, or algorithm is carriedout or executed. Some steps may be omitted in some embodiments or someoccurrences, or some steps may be executed more than once in a givenembodiment or occurrence.

When a single device or article is described herein, it will be readilyapparent that more than one device or article may be used in place of asingle device or article. Similarly, where more than one device orarticle is described herein, it will be readily apparent that a singledevice or article may be used in place of the more than one device orarticle.

The functionality or the features of a device may be alternativelyembodied by one or more other devices that are not explicitly describedas having such functionality or features. Thus, other embodiments of oneor more of the inventions need not include the device itself.

Techniques and mechanisms described or referenced herein will sometimesbe described in singular form for clarity. However, it should beappreciated that particular embodiments may include multiple iterationsof a technique or multiple instantiations of a mechanism unless notedotherwise. Process descriptions or blocks in figures should beunderstood as representing modules, segments, or portions of code whichinclude one or more executable instructions for implementing specificlogical functions or steps in the process. Alternate implementations areincluded within the scope of embodiments of the present invention inwhich, for example, functions may be executed out of order from thatshown or discussed, including substantially concurrently or in reverseorder, depending on the functionality involved, as would be understoodby those having ordinary skill in the art.

Conceptual Architecture

FIG. 1 is a side view of a variable-resistance exercise machine withwireless communication for smart device control and interactive softwareapplications 100, according to a preferred embodiment of the invention.According to the embodiment, an exercise machine 100 may have a stablebase 101 to provide a platform for a user to safely stand or move aboutupon.

Additional safety may be provided through the use of a plurality ofintegrally-formed or detachable side rails 102, for example havingsafety rails on the left and right sides (with respect to a user's pointof view) of exercise machine 100 to provide a stable surface for a userto grasp as needed. Additionally, side rails 102 may comprise aplurality of open regions 105 a-n formed to provide additional locationsfor a user to grasp or for the attachment of additional equipment suchas a user's smart device (not shown) through the use of a mountable orclamping case or mount. Formed or removable supports 106 a-n may be usedfor additional grip or mounting locations, for example to affix aplurality of tethers (not shown) for use in interaction with softwareapplications while a user is using exercise machine 100 (as describedbelow, referring to FIG. 3).

Exercise machine 100 may further comprise a rigid handlebar 103 affixedor integrally-formed on one end of exercise machine 100, for a user tohold onto while facing forward during use. Handlebar 103 may furthercomprise a stand or mount 104 for a user's smart device such as (forexample) a smartphone or tablet computer, so they may safely support andstow the device during use while keeping it readily accessible forinteraction (for example, to configure or interact with a softwareapplication they are using, or to select different applications, or tocontrol media playback during use, or other various uses). Handlebar 103may be used to provide a stable handle for a user to hold onto duringuse for safety or stability, as well as providing a rigid point for theuser to “push off” during use as needed, for example to begin using amoving treadmill surface (described below in FIG. 2). During use, a usermay also face away from handlebar 103, using exercise machine 100 in thereverse without their view or range of motion being obscured orobstructed by handlebar 103 (for example, for use with a virtual realitygame that requires a wide degree of movement from the user's hands forinteraction).

As illustrated, the base 101 of exercise machine 100 may be formed witha mild, symmetrical curvature, to better approximate the natural rangeof movement of a user's body during use. Common exercise machines suchas treadmills generally employ a flat surface, which can beuncomfortably during prolonged or vigorous use, and may causecomplications with multi-directional movement or interaction while auser's view is obscured, as with a headset (described below in FIG. 3).By incorporating a gradual curvature, a user's movements may feel morenatural and require less reorientation or accommodation to become fluidand proficient, and stress to the body may be reduced.

FIG. 3 is a diagram illustrating an exemplary system for a virtualreality or mixed reality enhanced exercise machine 100 with wirelesscommunication for smart device control and interactive softwareapplications using a smart device, illustrating the use of a pluralityof connected smart devices and tethers, and showing interaction via theuser's body as a control stick. According to the embodiment, a user 301may be standing, walking, or running on a variable-resistance exercisemachine 100 with wireless communication for smart device control andvirtual reality applications with a stable base 101 and two separatemoveable surfaces 203 a, 203 b for separate movement of the user's legs.Exercise machine 100 may have fixed handlebars with affixed orintegrally-formed controllers 305 a, 305 b for use as connected smartdevices for interaction, and support rails 201 a, 201 b for a user tohold onto or affix tethers for safety or interaction when needed. User401 may interact with software applications using a variety of means,including manual interaction via controller devices 305 a, 305 b thatmay be held in the hand for example to use as motion-input controldevices or (as illustrated) may be affixed or integrally-formed intoexercise machine 100. This may provide a user with traditional means ofinteracting with software applications while using exercise machine 100.Additionally, a user's body position or movement may be tracked and usedas input, for example via a plurality of tethers 304 a-n affixed tohandlebars 201 a, 201 b and a belt, harness or saddle 303 worn by user301, or using a headset device 302 that may track the position ormovement of a user's head as well as provide video (and optionallyaudio) output to the user, such as a virtual reality headset thatdisplays images while blocking the user's view of the outside world, oran augmented reality or mixed reality headset that combines presentedinformation with the user's view using transparent or semitransparentdisplays (for example, using transparent OLED displays, hologramdisplays, projected displays, or other various forms of overlaying adisplay within a user's normal field of vision without obstructing theuser's view). Body tracking may be used to recognize additional inputdata from user 301 (in addition to manual input via controllers 305 a,305 b), by tracking the position and movement of user 301 during use.For example, motion tracking within a headset device 302 may be used torecognize a variety of translational 310 or rotational 320 movement ofuser's 301 head, such as leaning to the side, or looking over theshoulder. Tethers 304 a-n may recognize a variety of movement of user's301 torso, such as leaning, crouching, sidestepping, or other bodymovement. This body tracking may then be utilized either as feedback torehab programs (for example, to track a user's posture for physicaltherapy coaching or exercises such as holding yoga poses) or inputsimilar to a control stick or joystick in manual controllerarrangements, for example by interpreting the user's entire body as the“stick” and processing their body movements as if they were stickmovements done manually (such as to control in-game character posture ormovement, or to direct movement in certain applications such as vehiclesimulations that may turn or accelerate in response to stick movements).

For example, a user 301 on exercise machine 100 may be playing a virtualreality skiing game or rehab program wherein they are given audio andvideo output via a headset 302 to immerse them in a virtual ski resort.When user 301 is not skiing, they may be able to use manual controls 305a, 305 b for such operations as selecting from an on-screen menu, ortyping text input such as to input their name or to chat with otherplayers using text. When they begin skiing within the game, user 301 maybe instructed in proper ski posture or technique, and may then use theirbody to control various aspects of their virtual skiing, such as leaningto the side 320 to alter their course and avoid trees or other skiers,or jumping 310 to clear rocks or gaps. Movement of their head may bedetected by a headset 302 and used to control their view independentlyof their body as it is tracked by tethers 304 a-n, allowing user 301 tolook around freely without interfering with their other controls. Inthis manner, the user's entire body may serve as an input control devicefor the game, allowing and encouraging them to use natural bodymovements to control their gameplay in an immersive manner while stillretaining the option to use more familiar manual control means asneeded. Alternatively, specific body functions such as hip twisting areused as user feedback for rehabilitating programs, including rehabgames.

FIG. 12 is a diagram illustrating an exemplary system 1200 for a virtualreality or mixed reality enhanced exercise machine 100, illustrating theuse of a plurality of optical sensors to detect body movement of a userduring use of an exercise machine. As above (with reference to FIG. 3),a user 301 may be standing, walking or running, sitting, or otherwisephysically active during use of an exercise machine 100. During use, theuser's position, posture, movement, cadence, technique, or any othermovement or position-related information may be detected, observed, ormeasured using a plurality of body movement sensors such as (forexample, including but not limited to) tethers 304 a-n that mayoptionally be affixed to handlebars 201 a-b or other features of anexercise machine 100, hardware sensors integrated into controllers 305a-b or a headset 302 the user may be using during exercise for virtualreality or mixed reality applications, or using a plurality of opticalsensors 1201 a-n that may be affixed to an exercise machine 100 oradjacent equipment, or that may be affixed to or positioned within anenvironment around exercise machine 100 to observe the user 301 duringuse. Optical sensors 1201 a-n may be used in a variety of configurationsor arrangements, such as using a single wide-angle sensor positioned toobserve a user's movement or posture from a particular angle (which maybe useful for coaching or physical therapy applications), or using morethan one sensor placed about a user to observe their movement inthree-dimensional space. A variety of hardware may be utilized inoptical sensors 1201 a-n, for example including (but not limited to) aninfrared or other optical camera that may directly observe the user'smovement, a structured-light emitter that projects a structured-lightgrid 1202 or other arrangement onto the user, exercise machine, orenvironment (and corresponding scanner or receiver that may observe theuser's movement through detected changes in the structured-lightprojection), or a light-field sensor that detects or measures depth toobserve a user's movement in three-dimensions. It should also beappreciated that various combination of optical sensors 1201 a-n may beutilized to achieve a desired effect, for example using both structuredlight and a light-field sensor to observe a user's movement in precisedetail in three dimensions. Additionally, some or all optical sensors1201 a-n utilized in some arrangements may be integrated into a user'sheadset 302 or an exercise machine 100 to provide “inside-out” trackingwhere tracking sensors are associated with the user rather than theenvironment, or they may be external devices as illustrated that may beintroduced to enhance an existing exercise machine or environment.

Utilizing an exercise machine 100 in this manner allows for a variety ofnovel forms of user interaction within virtual reality or mixed realityapplications. For example, a user's body movement during exercise may betracked in three dimensions and along or around various axes to recordmovement with six degrees of freedom (6DOF) comprising both translationalong, and rotation about, each of three spatial axes. This may be usedwith torso tracking as described above (referring to FIGS. 3-7) toproduce a 6DOF “torso joystick” virtual device that directs movement orother inputs within a software application. This may be used in a numberof ways, for example including but not limited to aiding exercisethrough interactive coaching (either with a human coach or usingsoftware to simulate a coach by providing feedback to detected usermovements), providing physical therapy, interacting with games or otherapplications during exercise, or using exercise combined with softwareinteraction for an immersive virtual reality or mixed realityexperience. For example, a user may control movement or expression of avirtual avatar or other user representation within a softwareapplication, such as using their own body movements to direct movementof a virtual character. Physical therapy or fitness coaching may utilizedetected movements to assist a user with improving their abilities ortechnique, or to measure progress. Social interaction applications mayutilize body movements during exercise, for example a chat or voice callapplication may utilize body movement as a form of nonverbal expressionsimilar to emoji or other icons. Safety may also be enhanced bycontrolling the operation of software in response to detected usermovements, for example displaying caution information or pausing anapplication if a user is detected to move outside a configured safetyparameter (such as stepping off a running treadmill, for example).

FIG. 8 is a block diagram of an exemplary system architecture 800 fornatural body interaction for mixed or virtual reality applications,according to a preferred embodiment of the invention. According to theembodiment, a composition server 801 comprising programming instructionsstored in a memory 11 and operating on a processor 12 of a computingdevice 10 (as described below, with reference to FIG. 13), may beconfigured to receive a plurality of input data from various connecteddevices. Such input devices may include (but are not limited to) avariety of hardware controller devices 804 (such as a gaming controller[such as GOJI PLAY™ controllers], motion tracking controller, ortraditional computer input devices such as a keyboard or mouse), aheadset device 803 such as an augmented reality or mixed or virtualreality headset (for example, OCULUS RIFT™, HTC VIVE™, SAMSUNG GEAR VR™,MICROSOFT MIXED REALITY™, or other headset devices), a variety offitness devices 805 (for example, fitness tracking wearable devices suchas FITBIT™, MICROSOFT BAND™, APPLE WATCH™, or other wearable devices),or a variety of body input 802 tracking devices or arrangements, such asusing a plurality of tethers attached to the environment and a harnessworn by a user, configured to track movement and position of the user'sbody.

Various input devices may be connected to composition server 801interchangeably as desired for a particular arrangement or use case, forexample a user may wish to use a controller 804 in each hand and aheadset 803, but omit the use of fitness devices 805 altogether. Duringoperation, composition server 801 may identify connected devices andload any stored configuration corresponding to a particular device ordevice type, for example using preconfigured parameters for use as adefault configuration for a new controller, or using historicalconfiguration for a headset based on previous configuration or use. Forexample, a user may be prompted (or may volunteer) to provideconfiguration data for a particular device, such as by selecting from alist of options (for example, “choose which type of device this is”, or“where are you wearing/holding this device”, or other multiple-choicetype selection), or composition server 801 may employ machine learningto automatically determine or update device configuration as needed. Forexample, during use, input values may be received that are determined tobe “out of bounds”, for example an erroneous sensor reading that mightindicate that a user has dramatically shifted position in a way thatshould be impossible (for example, an erroneous reading that appears toindicate the user has moved across the room and back again within afraction of a second, or has fallen through the floor, or other dataanomalies). These data values may be discarded, and configurationupdated to reduce the frequency of such errors in the future, increasingthe reliability of input data through use.

Composition server 801 may receive a wide variety of input data fromvarious connected devices, and by comparing against configuration datamay discard undesirable or erroneous readings as well as analyzereceived input data to determine more complex or fine-grainedmeasurements. For example, combining input from motion-sensingcontrollers 804 with a motion-sensing headset 803 may reveal informationabout how a user is moving their arms relative to their head or face,such as covering their face to shield against a bright light or anattack (within a game, for example), which might otherwise be impossibleto determine with any reliability using only the controllers themselves(as it may be observed that a user is raising their hands easily enough,but there is no reference for the position or movement of their head).These derived input values may then be combined into a single compositeinput data stream for use by various software applications, such asaugmented reality or mixed or virtual reality productivity applications(for example, applications that assist a user in performing manual tasksby presenting virtual information overlays onto their field of vision,or by playing audio directions to instruct them while observing theirbehavior through input devices, or other such applications), or mixed orvirtual reality applications or games, such as simulation games thattranslate a user's movement or position into in-game interaction, forexample by moving a user's in-game character or avatar based on theirphysical movements as received from input devices. In some arrangements,composition server 801 may operate such software applications in astandalone manner, functioning as a computer or gaming console asneeded. In other arrangements, composition server 801 may provide thecomposite data for use by an external computer 810, such as a connectedgaming console, mixed or virtual reality device, personal computer, or aserver operating via a network in the cloud (such as for online gamingarrangements, for example). In this manner, the composite data functionsof the embodiment may be utilized with existing hardware if desired, ormay be provided in a standalone package such as for demonstrations orpublic use, or for convenient setup using a single device to provide thefull interaction experience (in a manner similar to a household gamingconsole, wherein all the functions of computer components may beprepackaged and setup to minimize difficulty for a new user).

It should be appreciated that while reference is made to virtual realityapplications, a wide variety of use cases may be possible according tothe embodiment. For example, torso tracking may be used for fitness andhealth applications, to monitor a user's posture or gait while walking,without the use of additional virtual reality equipment or software. Insome arrangements, some or all interaction between a user and a softwareapplication may be nonvisual, and in some arrangements no display devicemay be present. In such an arrangement, a user may interact withsoftware entirely using feedback and movement of a worn harness 420 ortethers 304 a-n, using resistance or software-guided actuation oftethers 304 a-n (as described below, with reference to FIGS. 4-7) orother elements. In other arrangements, various combinations of displaydevices and other electronic devices may be used for a mixed-realitysetup, for example where a user's movement and interaction may be usedby software to incorporate elements of the physical world into a digitalrepresentation of the user or environment. For example, a user mayinteract with games or fitness applications, participate in social mediasuch as chat, calls, online discussion boards, social network postings,or other social content, or they may use body tracking to navigate userinterface elements of software such as a web browser or media player.Software used in this manner may not need to be specially-configured toutilize body tracking, for example to navigate a web browser a user'sbody movements or reactions to feedback may be processed by acomposition server 801 and mapped to generic inputs such as keystrokesor mouse clicks, for use in any standard software application withoutthe need for special configuration.

It should be further appreciated that while reference is made to atreadmill-type exercise machine 100, such an exercise machine isexemplary and any of a number of exercise machines may be utilizedaccording to the aspects disclosed herein, for example including (butnot limited to) a treadmill, a stationary bicycle, an ellipticalmachine, a rowing machine, or even non-electronic exercise equipmentsuch as a pull-up bar or weight machine. Traditional exercise equipmentmay be outfitted with additional components to facilitate virtualreality or mixed reality interaction according to the aspects disclosedherein, for example by affixing a plurality of tethers 304 a-n to aweight machine so that a user's movement during exercise may be used asinteraction as described below (with reference to FIGS. 3-7).

Detailed Description of Exemplary Embodiments

FIG. 2 is a top-down view of a variable-resistance exercise machine 100with wireless communication for smart device control and interactivesoftware applications, according to a preferred embodiment of theinvention. According to the embodiment, exercise machine 100 maycomprise a stable base 101 to provide a platform for a user to safelystand or move about upon. Exercise machine 100 may further compriseright 201 a and left 201 b hand rails for a user to brace against orgrip during use, to provide a stable support for safety as well as amounting point for external devices such as a plurality of tethers, asdescribed below with reference to FIG. 3. A plurality of steps 202 a-nmay be used to provide a user with a safe and easy means to approach ordismount exercise machine 100, as well as a nonmoving “staging area”where a user may stand while they configure operation or wait forexercise machine 100 to start operation. Unlike traditional treadmillmachines common in the art, exercise machine 100 may be made withgreater width to accommodate a wider range of free movement of a user'sentire body (whereas traditional treadmills are designed to bestaccommodate only a jogging or running posture, with minimal lateralmotion), and a plurality of separate moving surfaces 203 a-b may beutilized to provide multiple separate surfaces that may move and becontrolled independently of one another during use. For example, a usermay move each of their legs independently without resistance applied,with separate moving surfaces 203 a-b moving freely underfoot as a userapplies pressure during their movement. This may provide the illusion ofmovement to a user while in reality they remain stationary with respectto their surroundings. Another use may be multiple separate movingsurfaces 203 a-b, with separate speeds of movement or degrees ofresistance, so that as a user moves about during use they may experiencephysical feedback in the form of changing speed or resistance,indicating where they are standing or in what direction they are moving(for example, to orient a user wearing a virtual reality headset, asdescribed below with reference to FIG. 3). Moving surfaces 203 a-b maybe formed with a texture 204 to increase traction, which may improveuser safety and stability during use as well as improve the operation ofmoving surfaces 203 a-b for use in multidirectional movement (as theuser's foot is less likely to slide across a surface rather than takingpurchase and applying directional pressure to produce movement). Use ofmultiple, multidirectional moving surfaces 203 a-b may also be used invarious therapeutic or rehabilitation roles, for example to aid a userin developing balance or range of motion. For example, a user who isrecovering from an injury or surgery (such as a joint repair orreplacement surgery) may require regular physical therapy duringrecovery. Use of multidirectional moving surfaces 203 a-b along withappropriate guidance from a rehabilitation specialist or physicaltherapist (or optionally a virtual or remote coach using a softwareapplication) may make regular therapy more convenient and accessible tothe user, rather than requiring in-home care or regular visits to aclinic. For example, by enabling a therapist or coach to manually varythe movement and resistance of the moving surfaces 203 a-b, they canexamine a user's ability to overcome resistance to different movementssuch as at odd angles or across varying range of motion, to examine theuser's physical health or ability. By further varying the resistance itbecomes possible to assist the user with rehabilitation by providingtargeted resistance training to specific movements, positions, or musclegroups to assist in recovery and development of the user's abilities.

Exercise machine 100 may be designed without a control interfacecommonly utilized by exercise machines in the art, instead beingconfigured with any of a variety of wireless network interfaces such asWi-Fi or BLUETOOTH™ for connection to a user's smart device, such as asmartphone or tablet computer. When connected, a user may use a softwareapplication on their device to configure or direct the operation ofexercise machine 100, for example by manually configuring a variety ofoperation settings such as speed or resistance, or by interacting with asoftware application that automatically directs the operation ofexercise machine 100 without exposing the particular details ofoperation to a user. Additionally, communication may be bi-directional,with a smart device directing the operation of exercise machine 100 andwith exercise machine 100 providing input to a smart device based atleast in part on a user's activity or interaction. For example, a usermay interact with a game on their smart device, which directs theoperation of exercise machine 100 during play as a form of interactionwith, and feedback to, the user. For example, in a racing game, exercisemachine 100 may alter the resistance of moving surfaces 203 a-b as auser's speed changes within the game. In another example, a user may bemoving about on moving surfaces 203 a-b while playing a simulation orroleplaying game, and their movement may be provided to the connectedsmart device for use in controlling an in-game character's movement.Another example may be two-way interactive media control, wherein a usermay select media such as music for listening on their smart device, andthen while using exercise machine 100 their level of exertion (forexample, the speed at which they run or jog) may be used to provideinput to their smart device for controlling the playback of media. Forexample, if the user slows down music may be played slowly, distortingthe audio unless the user increases their pace. In this manner, exercisemachine 100 may be used interchangeably as a control and feedback deviceor both simultaneously, providing an immersive environment for a widevariety of software applications such as virtual reality, video games,fitness and health applications, or interactive media consumption.

FIG. 4 is a diagram of an exemplary hardware arrangement 400 for naturaltorso tracking and feedback for electronic interaction according to apreferred embodiment of the invention, illustrating the use of multipletethers 410 a-n and a movable torso harness 420. According to theembodiment, a plurality of tethers 410 a-n may be affixed orintegrally-formed as part of a handle or railing 430, such as handlebarsfound on exercise equipment such as a treadmill, elliptical trainer,stair-climbing machine, or the like. In alternate arrangements,specifically-designed equipment with integral tethers 410 a-n may beused, but it may be appreciated that a modular design with tethers 410a-n that may be affixed and removed freely may be desirable forfacilitating use with a variety of fitness equipment or structuralelements of a building, according to a user's particular use case orcircumstance. Tethers 410 a-n may then be affixed or integrally-formedto a torso harness 420, as illustrated in the form of a belt, that maybe worn by a user such that movement of their body affects tethers 410a-n and applies stress to them in a variety of manners. It should beappreciated that while a belt design for a torso harness 420 is shownfor clarity, a variety of physical arrangements may be used such asincluding (but not limited to) a vest, a series of harness-like strapssimilar to climbing or rappelling equipment, a backpack, straps designedto be worn on a user's body underneath or in place of clothing (forexample, for use in medical settings for collecting precise data) or aplurality of specially-formed clips or attachment points that may bereadily affixed to a user's clothing. Additionally, a torso harness 420may be constructed with movable parts, for example having an inner belt421 that permits a user some degree of motion within the harness 420without restricting their movement. Movement of inner belt 421 (or othermovable portions) may be measured in a variety of ways, such as usingaccelerometers, gyroscopes, or optical sensors, and this data may beused as interaction with software applications in addition to datacollected from tethers 410 a-n as described below. In some embodiments,a saddle-like surface on which a user may sit may be used, with motionof the saddle-like surface measured as described generally herein.

As a user moves, his or her body naturally shifts position andorientation. These shifts may be detected and measured via tethers 410a-n, for example by detecting patterns of tension or strain on tethers410 a-n to indicate body orientation, or by measuring small changes instrain on tethers 410 a-n to determine more precise movements such asbody posture while a user is speaking, or specific characteristics of auser's stride or gait. Additionally, through varying the quantity andarrangement of tethers 410 a-n, more precise or specialized forms ofmovement may be detected and measured (such as, for example, using aspecific arrangement of multiple tethers connected to a particular areaof a user's body to detect extremely small movements for medicaldiagnosis or fitness coaching). This data may be used as interactionwith software applications, such as for virtual reality applications asinput for a user to control a character in a game. In such anarrangement, when a user moves, this movement may be translated to anin-game character or avatar to convey a more natural sense ofinteraction and presence. For example, in a multiplayer roleplayinggame, this may be used to facilitate nonverbal communication andrecognition between players, as their distinct mannerisms and gesturesmay be conveyed in the game through detection of natural torso positionand movement. In fitness or health applications, this data may be usedto track and monitor a user's posture or ergonomic qualities, or toassist in coaching them for specific fitness activities such as holdinga pose for yoga, stretching, or proper running form during use with atreadmill. In medical applications, this data may be used to assist indiagnosing injuries or deficiencies that may require attention, such asby detecting anomalies in movement or physiological adaptations to anunrecognized injury (such as when a user subconsciously shifts theirweight off an injured foot or knee, without consciously realizing anissue is present).

Through various arrangements of tethers 410 a-n and tether sensors (asdescribed below, referring to FIGS. 5-7), it may be possible to enable avariety of immersive ways for a user to interact with softwareapplications, as well as to receive haptic feedback from applications.For example, by detecting rotation, tension, stress, or angle of tethersa user may interact with applications such as virtual reality games orsimulations, by using natural body movements and positioning such asleaning, jumping, crouching, kneeling, turning, or shifting their weightin various directions to trigger actions within a software applicationconfigured to accept torso tracking input. By applying haptic feedbackof varying form and intensity (as is described in greater detail below,referring to FIG. 5), applications may provide physical indication to auser of software events, such as applying tension to resist movement,pulling or tugging on a tether to move or “jerk” a user in a direction,or varying feedback to multiple tethers such as tugging and releasing invarying order or sequence to simulate more complex effects such as (forexample, in a gaming use case) explosions, riding in a vehicle, orwalking through foliage.

FIG. 5 is a diagram illustrating a variety of alternate tetherarrangements. According to various use cases and hardware arrangements,tethers 410 a-n may utilize a variety of purpose-driven designs asillustrated. For example, a “stretchable” tether 510 may be used tomeasure strain during a user's movement, as the tether 510 is stretchedor compressed (for example, using piezoelectric materials and measuringelectrical changes). Such an arrangement may be suitable for precisemeasurements, but may lack the mechanical strength or durability forgross movement detection or prolonged use. An alternate construction mayutilize a non-deforming tether 520 such as a steel cable or similarnon-stretching material. Instead of measuring strain on the tether 520,instead tether 520 may be permitted a degree of movement within anenclosure 522 (for example, an attachment point on a torso harness 420or handlebar 430), and the position or movement 521 of the tether 520may be measured such as via optical sensors. In a third exemplaryarrangement, a tether 530 may be wound about an axle or pulley 531, andmay be let out when force is applied during a user's movement. Rotationof the pulley 531 may be measured, or alternately a tension device suchas a coil spring may be utilized (not shown) and the tension or strainon that device may be measured as tether 530 is extended or retracted.In this manner, it may be appreciated that a variety of mechanical meansmay be used to facilitate tethers and attachments for use in detectingand measuring natural torso position and movement, and it should beappreciated that a variety of additional or alternate hardwarearrangements may be utilized according to the embodiments disclosedherein.

Additionally, through the use of various hardware construction itbecomes possible to utilize both “passive” tethers that merely measuremovement or strain, as well as “active” tethers that may applyresistance or movement to provide haptic feedback to a user. Forexample, in an arrangement utilizing a coiled spring or pulley 531, thespring or pulley 531 may be wound to retract a tether and direct orimpede a user's movement as desired. In this manner, various new formsof feedback-based interaction become possible, and in virtual realityuse cases user engagement and immersion are increased through morenatural physical feedback during their interaction.

By applying various forms and intensities of feedback using varioustether arrangements, a variety of feedback types may be used to providehaptic output to a user in response to software events. For example,tension on a tether may be used to simulate restrained movement such aswading through water or dense foliage, walking up an inclined surface,magnetic or gravitational forces, or other forms of physical resistanceor impedance that may be simulated through directional ornon-directional tension. Tugging, retracting, or pulling on a tether maybe used to simulate sudden forces such as recoil from gunfire,explosions, being grabbed or struck by a software entity such as anobject or character, deploying a parachute, bungee jumping, sliding orfalling, or other momentary forces or events that may be conveyed with atugging or pulling sensation. By utilizing various patterns of hapticfeedback, more complex events may be communicated to a user, such asriding on horseback or in a vehicle, standing on the deck of a ship atsea, turbulence in an aircraft, weather, or other virtual events thatmay be represented using haptic feedback. In this manner, virtualenvironments and events may be made more immersive and tangible for auser, both by enabling a user to interact using natural body movementsand positioning, as well as by providing haptic feedback in a mannerthat feels natural and expected to the user. For example, if a user iscontrolling a character in a gaming application through a first-personviewpoint, it would seem natural that when their character is struckthere would be a physical sensation corresponding to the event; however,this is not possible with traditional interaction devices, detractingfrom any sense of immersion or realism for the user. By providing thisphysical sensation alongside the virtual event, the experience becomesmore engaging and users are encouraged to interact more naturally astheir actions results in natural and believable feedback, meeting theirsubconscious expectations and avoiding excessive “immersion-breaking”moments, which in turn reduces the likelihood of users adopting unusualbehaviors or unhealthy posture as a result of adapting to limitedinteraction schema.

Haptic feedback may be provided to notify a user of non-gaming events,such as for desktop notifications for email or application updates, orto provide feedback on their posture for use in fitness or healthcoaching. For example, a user may be encouraged to maintain a particularstance, pose, or posture while working or for a set length of time (forexample, for a yoga exercise application), and if their posture deviatesfrom an acceptable range, feedback is provided to remind them to adjusttheir posture. This may be used in sports, fitness, health, or ergonomicapplications that need not utilize other aspects of virtual reality andmay operate as traditional software applications on nonspecializedcomputing hardware. For example, a user at their desk may use anergonomic training application that monitors their body posturethroughout the work day and provides haptic reminders to correct poorposture as it is detected, helping the user to maintain a healthyworking posture to reduce fatigue or injuries due to poor posture (forexample, repetitive-stress injuries that may be linked to poor posturewhile working at a computer).

FIG. 6 is a diagram of an additional exemplary hardware arrangement 600for natural torso tracking and feedback for electronic interactionaccording to a preferred embodiment of the invention, illustrating theuse of angle sensors 612, 621 a-n to detect angled movement of a tether620. According to one exemplary arrangement, a tether 610 may be affixedto or passed through a rotating joint such as a ball bearing 611 orsimilar, to permit free angular movement. During movement, the angularmovement or deflection 612 of a protruding bar, rod, or tether segment613 may be measured (for example, using optical, magnetic, or othersensors) to determine the corresponding angle of tether 610. In thismanner, precise angle measurements may be collected without impedingrange of motion or introducing unnecessary mechanical complexity.

In an alternate hardware arrangement, the use of angle sensors 621 a-nenables tracking of a vertical angle of a tether 620, to detect andoptionally measure vertical movement or orientation of a user's torso.When tether 620 contacts a sensor 621 a-n, this may be registered andused to detect a general vertical movement (that is, whether the tetheris angled up or down). For more precise measurements, the specifichardware construction of a sensor 621 a-n may be varied, for exampleusing a pressure-sensing switch to detect how much force is applied anduse this measurement to determine the corresponding angle (as may bepossible given a tether 620 of known construction). It should beappreciated that various combinations of hardware may be used to providea desired method or degree of angle detection or measurement, forexample using a conductive tether 620 and a capacitive sensor 621 a-n todetect contact, or using a mechanical or rubber-dome switch (as arecommonly used in keyboard construction) to detect physical contactwithout a conductive tether 620.

The use of angle detection or measurement may expand interactionpossibilities to encompass more detailed and natural movements of auser's body. For example, if a user crouches, then all tethers 410 a-nmay detect a downward angle simultaneously. Additionally, data precisionor availability may be enhanced by combining input from multipleavailable sensors when possible (for example, utilizing adaptivesoftware to collect data from any sensors that it detects, withoutrequiring specific sensor types for operation), for example by combiningdata from tethers 410 a-n and hardware sensors such as an accelerometeror gyroscope, enabling multiple methods of achieving similar or variedtypes or precision levels of position or movement detection. Similarly,when a user jumps then all tethers may detect an upward anglesimultaneously. However, if a user leans in one direction, it may beappreciated that not all tethers 410 a-n will detect the same angle. Forexample, tethers 410 a-n in the direction the user is leaning may detecta downward angle, while those on the opposite side would detect anupward angle (due to the orientation of the user's torso and thus a worntorso harness 420). In this manner, more precise torso interaction maybe facilitated through improved detection and recognition of orientationand movement. Additionally, it may be appreciated that sensors 621 a-nmay be utilized for other angle measurements, such as to detecthorizontal angle. For example, if a user is wearing a non-rotating torsoharness 420, when they twist their body a similar stress may be appliedto all attached tethers 410 a-n. Without angle detection the precisenature of this movement will be vague, but with horizontal angledetection it becomes possible to recognize that all tethers 410 a-n arebeing strained in a similar direction (for example, in a clockwisepattern when viewed from above, as a user might view tethers 410 a-nduring use), and therefore interpret the interaction as a twistingmotion (rather than, for example, a user squatting or kneeling, whichmight apply a similar stress to the tethers 410 a-n but would havedifferent angle measurements).

FIG. 7 is a diagram illustrating an exemplary hardware arrangement of anapparatus for natural torso tracking and feedback for electronicinteraction according to a preferred embodiment of the invention,illustrating the use of multiple tethers 410 a-n and a movable torsoharness 420 comprising a plurality of angle sensors 701 a-n positionedwithin the movable torso harness 420. According to the embodiment, aplurality of tethers 410 a-n may be affixed or integrally-formed as partof a handle or railing 430, such as handlebars found on exerciseequipment such as a treadmill, elliptical trainer, stair-climbingmachine, or the like. In alternate arrangements, specifically-designedequipment with affixed or integral tethers 410 a-n may be used, but itmay be appreciated that a modular design with tethers 410 a-n that maybe affixed and removed freely may be desirable for facilitating use witha variety of fitness equipment or structural elements of a building,according to a user's particular use case or circumstance as well asweight-holding strength of the tethers. Tethers 410 a-n may then beaffixed or integrally-formed to angle sensors 701 a-n placed within orintegrally-formed as a component of torso harness 420 (as illustrated inthe form of a belt) that may be worn by a user such that movement oftheir body affects tethers 410 a-n and applies detectable or measurablestress to tethers 410 a-n and angular motion to angle sensors 701 a-n.In this manner, it may be appreciated that angle sensors 701 a-n may beutilized as integral or removable components of a torso harness 420, asan alternative arrangement to utilizing angle sensors 701 a-n placed orformed within railings 430 or other equipment components connected todistal ends of tethers 410 a-n (with respect to the user's torso).According to various embodiments, sensors may be placed optionally on abelt, vest, harness, or saddle-like surface or at attachment points onsafety railings, or indeed both.

FIG. 9 is a block diagram of an exemplary system architecture 900 of anexercise machine 100 being connected over local connections to asmartphone or computing device 930, an output device other than a phone910, and a server over a network 940, according to a preferred aspect.An exercise machine 100 may connect over a network 920, which may be theInternet, a local area connection, or some other network used fordigital communication between devices, to a server 940. Such connectionmay allow for two-way communication between a server 940 and an exercisemachine 800. An exercise machine 100 may also be connected over anetwork 920 to a smartphone or computing device 930, or may be connecteddirectly to a smartphone or computing device 930 either physically orwirelessly such as with Bluetooth connections. An exercise machine 100also may be connected to an output device 910 which may displaygraphical output from software executed on an exercise machine 100,including Mixed or virtual reality software, and this device may bedifferent from a smartphone or computing device 930 or in someimplementations may in fact be a smartphone or computing device 930. Aremote server 940 may contain a data store 941, and a user verificationcomponent 942, which may contain typical components in the art used forverifying a user's identity from a phone connection or deviceconnection, such as device ID from a smartphone or computing device orlogging in with a user's social media account.

FIG. 10 is a diagram of an exemplary hardware arrangement of a smartphone or computing device 1030 executing software 1010 and communicatingover a network 1020, according to a preferred aspect. In an exemplarysmart phone or computing device 1030, key components include a wirelessnetwork interface 1031, which may allow connection to one or a varietyof wireless networks including Wi-Fi and Bluetooth; a processor 1032,which is capable of communicating with other physical hardwarecomponents in the computing device 1030 and running instructions andsoftware as needed; system memory 1033, which stores temporaryinstructions or data in volatile physical memory for recall by thesystem processor 1032 during software execution; and a display device1034, such as a Liquid Crystal Display (LCD) screen or similar, withwhich a user may visually comprehend what the computing device 1030 isdoing and how to interact with it. It may or may not be a touch enableddisplay, and there may be more components in a computing device 1030,beyond what are crucially necessary to operate such a device at all.Software 1010 operating on a processor 1033 may include a mixed orvirtual reality application, a user verification system, or othersoftware which may communicate with a network-enabled server 1040 andexercise machine 100 software for the purposes of enhanced mixed orvirtual reality.

FIG. 11 is a block diagram of a method of mixed or virtual realitysoftware operating to receive input through different sources, and sendoutput to devices, according to a preferred aspect. Mixed or virtualreality software which may be run on a phone or computing device 1030 oranother device, outputs data to a visual device for the purpose ofgraphically showing a user what they are doing in the software 1110.Such display may be a phone display 1034, or a separate display devicesuch as a screen built into an exercise machine 100 or connected someother way to the system, or both display devices. During softwareexecution, user input may be received either through buttons 1130 on theexercise machine 100, 1120, or through input from a belt-like harness420, such as user orientation or movements. Such received data may besent 1140 to either a mobile smart phone or computing device 1030, or toa server 1040 over a network 1020, or both, for processing, storage, orboth. Data may be stored on a server with a data store device 1041 andmay be processed for numerous uses including user verification with auser verification component 1042. Data may be processed either bysoftware running on an exercise machine 100, a smart phone or computingdevice 1030, or some other connected device which may be running mixedor virtual reality software, when input is received from a user usingeither buttons on an exercise machine 100, a belt-like harness 420, orboth, and optionally using hardware features of an exercise machine 100such as handlebars, pedals, or other features in mixed or virtualreality software for tasks such as representing movement in asimulation.

FIG. 17 is a block diagram of an exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a stationarybicycle 1700 with hand controls on the handles 1720, and a belt-likeharness attachment 420. A stationary exercise bicycle device 1700, whichmay be of any particular design including a reclining, sitting, or evenunicycle-like design, possesses two pedals 1730 as is common forstationary exercise bicycles of all designs. On handlebars of astationary exercise bicycle may exist buttons and controls 1720 forinteracting with a virtual reality or mixed reality augmented piece ofsoftware, allowing a user to press buttons in addition to or instead ofpedaling, to interact with the software. A belt-like harness attachment420 is attached via a mechanical arm 1710 to a stationary exercisebicycle 1700, which may monitor motion and movements from a user duringthe execution of virtual reality software. A mechanical arm 1710 mayhave an outer shell composed of any material, the composition of whichis not claimed, but must have hinges 1711, 1712, 1713 which allow fordynamic movement in any position a user may find themselves in, andangular sensors inside of the arm at the hinge-points 1711, 1712, 1713for measuring the movement in the joints and therefore movement of theuser. A stationary bicycle device 1700 may also have a pressure sensorin a seat 1740, the sensor itself being of no particularly novel designnecessarily, to measure pressure from a user and placement of saidpressure, to detect movements such as leaning or sitting lop-sidedrather than sitting evenly on the seat.

FIG. 18 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine 1800, illustrating the use of atreadmill exercise machine 100 a vest-type harness 1820 with a pluralityof pistons 1811 to provide a hardware-based torso joystick withfull-body tracking. According to this embodiment, a treadmill or otherexercise machine 100 may comprise a plurality of rigid side rails 102for a user to grip for support as needed during use (for example, as abalance aid or to assist getting on the machine and setting up otherequipment properly) as well as a rigid stand or mount 104 for a user'ssmartphone or other computing device, that may be used to operate avirtual reality or mixed reality software application. Exercise machine100 may further comprise a jointed arm 1810 or similar assembly that maybe integrally-formed or removably affixed to or installed upon exercisemachine 100. Arm 1810 may utilize a plurality of pistons 1811 to providefor movement during use in order to follow the movement's of a user'sbody, as well as to provide tension or resistance to motion whenappropriate (for example, to resist a user's movements or to providefeedback) and motion detection of a user's movement during use,according to various aspects described previously (referring to FIGS.3-7, for example) by measuring movement of a piston 1811 or arm 1810 andoptionally applying tension or resistance to piston 1811 to retardmovement of arm 1810 and constrain user movement or simulate specificforms of physical feedback. For example, if a user is moving an avatarin a virtual reality software application, when the avatar encounters anobstacle such as another avatar, object, or part of the environment,resistance may be applied to piston 1811 to prevent the user from movingfurther, so that their avatar is effectively prevented from movingthrough the obstacle and thereby facilitating the immersive experienceof a solid object in a virtual environment. Additional arms may be usedfor a user's limbs 1821 and may incorporate straps 1822 to be affixabout a user's arm, wrist, or other body part (for example, when placedthrough an appropriate arm or limb hole 1823 in vest harness 1820 whileworn), to incorporate more detailed movement tracking of a user's armsand/or legs rather than just torso-based tracking. A vest-type harness1820 may be affixed to jointed arm 1810 using a movable joint 1812 suchas a ball joint (for example) and used in place of a belt 420, to allowfor more natural movement or to provide greater area upon which to affixadditional arms 1821, pistons 1811, or any of a variety of sensors, forexample such as accelerometers or gyroscopes for detecting bodyorientation (not all optional sensors are shown for the sake ofclarity). For example, a vest 1820 may have integrated feedbackactuators for use in first-person software applications to simulateimpacts or recoil, or it may incorporate heating or cooling elements tosimulate different virtual environments while worn. Additionally, vest1820 may incorporate electrical connectors 1824 for various peripheraldevices such as controllers 305 a-b or a headset 302, reducing the riskof tangles or injury by keeping cables short and close to the user sothey cannot cause issues during movement or exercise.

FIG. 19 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine, illustrating the use of a stationarybicycle 1900 with a vest-type harness 1820 with a plurality of strainsensors 1911 and tethers 1912, according to an aspect of the invention.According to this embodiment, rather than a jointed arm 1810 and pistons1811, a solid flexible arm 1910 may be used to detect user movementwhile positioned on a seat 1902 to use exercise machine 100, for examplewhile the user is seated to use pedals 1901 on a stationary bike orelliptical training machine. Through a plurality of strain gauges 1911that detect the flexion or extension of the solid arm. Tethers 1912 maybe used for either movement tracking or providing feedback to a user, orboth, and may optionally be connected or routed through joints orinterconnects 1913 to allow for a greater variety of attachment optionsas well more precise feedback (for example, by enabling multiple anglesfrom which a tether 1912 may apply force, to precisely simulatedifferent effects). Additional arms may be used for a user's limbs 1921and may incorporate straps 1922 to be affix about a user's arm, wrist,or other body part, to incorporate more detailed movement tracking of auser's arms and/or legs rather than just torso-based tracking.Additional arms 1921 may also incorporate additional tethers 1912 andstrain sensors 1911 to track movement and apply feedback to specificbody parts during use, further increasing precision and user immersion.A vest-type harness 1820 may be used in place of a belt 420, to allowfor more natural movement or to provide greater area upon which to affixadditional arms 1921, tether 1912, or any of a variety of sensors, forexample such as accelerometers or gyroscopes for detecting bodyorientation (not all optional sensors are shown for the sake ofclarity). For example, a vest 1820 may have integrated feedbackactuators for use in first-person software applications to simulateimpacts or recoil, or it may incorporate heating or cooling elements tosimulate different virtual environments while worn. Additionally, vest1820 may incorporate electrical connectors 1914 for various peripheraldevices such as controllers 305 a-b or a headset 302, reducing the riskof tangles or injury by keeping cables short and close to the user sothey cannot cause issues during movement or exercise.

FIG. 20 is a flow diagram illustrating an exemplary method 2000 foroperating a virtual and mixed-reality enhanced exercise machine,according to one aspect. According to the aspect, a user may wear 2001 atorso harness such as a belt 420 or vest 1820 harness, while they engagein the use 2002 of an exercise machine 100. While using the exercisemachine 100, the user's movements may be detected and measured 2003through the use of a plurality of body movement sensors such as (forexample, including but not limited to) strain sensors 1911, tethers 410a-c, 1912, pistons 1811, or optical sensors 1201 a-n. These measureduser movements may then be mapped by a composition server 801 tocorrespond to a plurality of movement inputs of a virtual joystickdevice 2004. These virtual joystick inputs may then be transmitted 2005to a software application, for example a virtual reality or mixedreality application operating on a user device such as (for example,including but not limited to) a smartphone 930, personal computingdevice, or headset 302. Composition server 801 may then receive feedbackfrom the software application 2006, and may direct the operation of aplurality of feedback devices such as tethers 410 a-c, 1912 or pistons1811 to resist or direct the user's movement 2007 to provide physicalfeedback to the user based on the received software feedback.

FIG. 21 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine 2100, illustrating the use of arotating platform, a waist belt 2110 and joints 2120 a-c providing fullrange of motion. As shown in view 2, a virtual reality or mixed realityexercise machine 2100 may comprise an exercise machine such as (forexample) a treadmill 2130 that has a movable or pivotable base 2132, forexample via a turnstile 2131 or similar joint for allowing movement ofthe entire exercise machine 2100 while base 2132 remains stationary orfixed to the floor. A waist belt 2110 such as a padded hip belt or asupport belt (such as those used for weight lifting or other physicalactivities, or those used for physical therapy or other medical uses)may be utilized with belt straps 2111 and buckle 2112 to fasten about auser's waist. Waist belt 2110 may be attached to exercise machine 2100using a plurality of ball or similar joints 2120 a-c configured to allowfor full range of motion, enabling complex user movements and postureand allowing the use of full freedom of movement when interacting withsoftware applications. A plurality of pistons 1811 or other means may beused, as described previously, to restrict or direct user movement or toprovide feedback, enabling a variety of interaction and feedback optionsas well as providing a means to control and manipulate user movementboth for immersion in virtual or mixed reality applications as well asfor a variety of medical or therapeutic uses, such as preventing usersfrom exceeding recommended range of motion during physical therapy orpreventing movement past safety parameters.

FIG. 22 is a diagram illustrating a top view of a waist belt withstraps, buckle, and ball joint providing full range of motion. A waistbelt 2110 such as a padded hip belt or a support belt (such as thoseused for weight lifting or other physical activities, or those used forphysical therapy or other medical uses) may be utilized with belt straps2111 and buckle 2112 to fasten about a user's waist. Waist belt 2110 maybe attached to exercise machine 2100 using a ball or similar joint 2120c configured to allow for full range of motion, enabling complex usermovements and posture and allowing the use of full freedom of movementwhen interacting with software applications.

FIG. 23 is a diagram of another exemplary virtual reality or mixedreality enhanced exercise machine 2200, illustrating the use of amovable exercise machine 2200 with a rotating platform. A virtualreality or mixed reality exercise machine 2200 may comprise an exercisemachine such as (for example) a stepping machine 2210 that has movablefoot pedals 2211 a-b for simulating stairs or other vertical movement,railing 2212 for a user to grip for safety and support, and a screen orcontrol interface 2213 for directing the operation of stepping machine2210 or interacting with software applications (such as, for example,virtual reality or mixed reality applications). Exercise machine 2200may further comprise a movable or pivotable base 2132, for example via arotating platform 2131 or similar joint for allowing movement of theentire exercise machine 2210 while base 2132 remains stationary or fixedto the floor. This enables a user to continue using exercise machine2200 while being able to adjust the orientation of the machine (forexample, to gradually adjust to accommodate changes in lighting or otherenvironmental features, or to move the machine manually to face towardor away from other items in the area such as a TV, speakers, or otherusers using other exercise equipment). This also provides a means forexercise machine 2200 to be moved during the use of virtual or mixedreality software applications, for example as a form of interaction orimmersion by moving the user during a game or providing feedback in theform of small movements to simulate changes in a virtual environment.

FIG. 24 is a diagram illustrating a top view of pivoting foot pedals2211 a-b. Foot pedals 2211 a-b may be pivotable or otherwise movable, toprovide additional degrees of motion for user adjustment, immersion, andfeedback. For example, a user may continue to face in the same directionwhile rotating platform 2131 and foot pedals 2211 a-b pivot, changingthe angle of resistance as the user continues to step on pedals 2211a-b, such as to simulate walking up an inclined floor or staircase.Small movements of rotating platform 2131 and foot pedals 2211 a-b maybe used to simulate features such as rough or uneven terrain or smallobstacles, or to provide feedback to the user such as to simulate anearthquake, the pitching deck of a ship, a moving floor, or othersimulate stimuli.

Hardware Architecture

Generally, the techniques disclosed herein may be implemented onhardware or a combination of software and hardware. For example, theymay be implemented in an operating system kernel, in a separate userprocess, in a library package bound into network applications, on aspecially constructed machine, on an application-specific integratedcircuit (ASIC), or on a network interface card.

Software/hardware hybrid implementations of at least some of theembodiments disclosed herein may be implemented on a programmablenetwork-resident machine (which should be understood to includeintermittently connected network-aware machines) selectively activatedor reconfigured by a computer program stored in memory. Such networkdevices may have multiple network interfaces that may be configured ordesigned to utilize different types of network communication protocols.A general architecture for some of these machines may be describedherein in order to illustrate one or more exemplary means by which agiven unit of functionality may be implemented. According to specificembodiments, at least some of the features or functionalities of thevarious embodiments disclosed herein may be implemented on one or moregeneral-purpose computers associated with one or more networks, such asfor example an end-user computer system, a client computer, a networkserver or other server system, a mobile computing device (e.g., tabletcomputing device, mobile phone, smartphone, laptop, or other appropriatecomputing device), a consumer electronic device, a music player, or anyother suitable electronic device, router, switch, or other suitabledevice, or any combination thereof. In at least some embodiments, atleast some of the features or functionalities of the various embodimentsdisclosed herein may be implemented in one or more virtualized computingenvironments (e.g., network computing clouds, virtual machines hosted onone or more physical computing machines, or other appropriate virtualenvironments).

Referring now to FIG. 13, there is shown a block diagram depicting anexemplary computing device 10 suitable for implementing at least aportion of the features or functionalities disclosed herein. Computingdevice 10 may be, for example, any one of the computing machines listedin the previous paragraph, or indeed any other electronic device capableof executing software- or hardware-based instructions according to oneor more programs stored in memory. Computing device 10 may be configuredto communicate with a plurality of other computing devices, such asclients or servers, over communications networks such as a wide areanetwork a metropolitan area network, a local area network, a wirelessnetwork, the Internet, or any other network, using known protocols forsuch communication, whether wireless or wired.

In one embodiment, computing device 10 includes one or more centralprocessing units (CPU) 12, one or more interfaces 15, and one or morebusses 14 (such as a peripheral component interconnect (PCI) bus). Whenacting under the control of appropriate software or firmware, CPU 12 maybe responsible for implementing specific functions associated with thefunctions of a specifically configured computing device or machine. Forexample, in at least one embodiment, a computing device 10 may beconfigured or designed to function as a server system utilizing CPU 12,local memory 11 and/or remote memory 16, and interface(s) 15. In atleast one embodiment, CPU 12 may be caused to perform one or more of thedifferent types of functions and/or operations under the control ofsoftware modules or components, which for example, may include anoperating system and any appropriate applications software, drivers, andthe like.

CPU 12 may include one or more processors 13 such as, for example, aprocessor from one of the Intel, ARM, Qualcomm, and AMD families ofmicroprocessors. In some embodiments, processors 13 may includespecially designed hardware such as application-specific integratedcircuits (ASICs), electrically erasable programmable read-only memories(EEPROMs), field-programmable gate arrays (FPGAs), and so forth, forcontrolling operations of computing device 10. In a specific embodiment,a local memory 11 (such as non-volatile random access memory (RAM)and/or read-only memory (ROM), including for example one or more levelsof cached memory) may also form part of CPU 12. However, there are manydifferent ways in which memory may be coupled to system 10. Memory 11may be used for a variety of purposes such as, for example, cachingand/or storing data, programming instructions, and the like. It shouldbe further appreciated that CPU 12 may be one of a variety ofsystem-on-a-chip (SOC) type hardware that may include additionalhardware such as memory or graphics processing chips, such as a QUALCOMMSNAPDRAGON™ or SAMSUNG EXYNOS™ CPU as are becoming increasingly commonin the art, such as for use in mobile devices or integrated devices.

As used herein, the term “processor” is not limited merely to thoseintegrated circuits referred to in the art as a processor, a mobileprocessor, or a microprocessor, but broadly refers to a microcontroller,a microcomputer, a programmable logic controller, anapplication-specific integrated circuit, and any other programmablecircuit.

In one embodiment, interfaces 15 are provided as network interface cards(NICs). Generally, NICs control the sending and receiving of datapackets over a computer network; other types of interfaces 15 may forexample support other peripherals used with computing device 10. Amongthe interfaces that may be provided are Ethernet interfaces, frame relayinterfaces, cable interfaces, DSL interfaces, token ring interfaces,graphics interfaces, and the like. In addition, various types ofinterfaces may be provided such as, for example, universal serial bus(USB), Serial, Ethernet, FIREWIRE™, THUNDERBOLT™, PCI, parallel, radiofrequency (RF), BLUETOOTH™, near-field communications (e.g., usingnear-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fastEthernet interfaces, Gigabit Ethernet interfaces, Serial ATA (SATA) orexternal SATA (ESATA) interfaces, high-definition multimedia interface(HDMI), digital visual interface (DVI), analog or digital audiointerfaces, asynchronous transfer mode (ATM) interfaces, high-speedserial interface (HSSI) interfaces, Point of Sale (POS) interfaces,fiber data distributed interfaces (FDDIs), and the like. Generally, suchinterfaces 15 may include physical ports appropriate for communicationwith appropriate media. In some cases, they may also include anindependent processor (such as a dedicated audio or video processor, asis common in the art for high-fidelity AN hardware interfaces) and, insome instances, volatile and/or non-volatile memory (e.g., RAM).

Although the system shown in FIG. 13 illustrates one specificarchitecture for a computing device 10 for implementing one or more ofthe inventions described herein, it is by no means the only devicearchitecture on which at least a portion of the features and techniquesdescribed herein may be implemented. For example, architectures havingone or any number of processors 13 may be used, and such processors 13may be present in a single device or distributed among any number ofdevices. In one embodiment, a single processor 13 handles communicationsas well as routing computations, while in other embodiments a separatededicated communications processor may be provided. In variousembodiments, different types of features or functionalities may beimplemented in a system according to the invention that includes aclient device (such as a tablet device or smartphone running clientsoftware) and server systems (such as a server system described in moredetail below).

Regardless of network device configuration, the system of the presentinvention may employ one or more memories or memory modules (such as,for example, remote memory block 16 and local memory 11) configured tostore data, program instructions for the general-purpose networkoperations, or other information relating to the functionality of theembodiments described herein (or any combinations of the above). Programinstructions may control execution of or comprise an operating systemand/or one or more applications, for example. Memory 16 or memories 11,16 may also be configured to store data structures, configuration data,encryption data, historical system operations information, or any otherspecific or generic non-program information described herein.

Because such information and program instructions may be employed toimplement one or more systems or methods described herein, at least somenetwork device embodiments may include nontransitory machine-readablestorage media, which, for example, may be configured or designed tostore program instructions, state information, and the like forperforming various operations described herein. Examples of suchnontransitory machine-readable storage media include, but are notlimited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks; magneto-optical mediasuch as optical disks, and hardware devices that are speciallyconfigured to store and perform program instructions, such as read-onlymemory devices (ROM), flash memory (as is common in mobile devices andintegrated systems), solid state drives (SSD) and “hybrid SSD” storagedrives that may combine physical components of solid state and hard diskdrives in a single hardware device (as are becoming increasingly commonin the art with regard to personal computers), memristor memory, randomaccess memory (RAM), and the like. It should be appreciated that suchstorage means may be integral and non-removable (such as RAM hardwaremodules that may be soldered onto a motherboard or otherwise integratedinto an electronic device), or they may be removable such as swappableflash memory modules (such as “thumb drives” or other removable mediadesigned for rapidly exchanging physical storage devices),“hot-swappable” hard disk drives or solid state drives, removableoptical storage discs, or other such removable media, and that suchintegral and removable storage media may be utilized interchangeably.Examples of program instructions include both object code, such as maybe produced by a compiler, machine code, such as may be produced by anassembler or a linker, byte code, such as may be generated by forexample a JAVA™ compiler and may be executed using a Java virtualmachine or equivalent, or files containing higher level code that may beexecuted by the computer using an interpreter (for example, scriptswritten in Python, Perl, Ruby, Groovy, or any other scripting language).

In some embodiments, systems according to the present invention may beimplemented on a standalone computing system. Referring now to FIG. 14,there is shown a block diagram depicting a typical exemplaryarchitecture of one or more embodiments or components thereof on astandalone computing system. Computing device 20 includes processors 21that may run software that carry out one or more functions orapplications of embodiments of the invention, such as for example aclient application 24. Processors 21 may carry out computinginstructions under control of an operating system 22 such as, forexample, a version of MICROSOFT WINDOWS™ operating system, APPLE MACOS™or iOS™ operating systems, some variety of the Linux operating system,ANDROID™ operating system, or the like. In many cases, one or moreshared services 23 may be operable in system 20, and may be useful forproviding common services to client applications 24. Services 23 may forexample be WINDOWS™ services, user-space common services in a Linuxenvironment, or any other type of common service architecture used withoperating system 21. Input devices 28 may be of any type suitable forreceiving user input, including for example a keyboard, touchscreen,microphone (for example, for voice input), mouse, touchpad, trackball,or any combination thereof. Output devices 27 may be of any typesuitable for providing output to one or more users, whether remote orlocal to system 20, and may include for example one or more screens forvisual output, speakers, printers, or any combination thereof. Memory 25may be random-access memory having any structure and architecture knownin the art, for use by processors 21, for example to run software.Storage devices 26 may be any magnetic, optical, mechanical, memristor,or electrical storage device for storage of data in digital form (suchas those described above, referring to FIG. 13). Examples of storagedevices 26 include flash memory, magnetic hard drive, CD-ROM, and/or thelike.

In some embodiments, systems of the present invention may be implementedon a distributed computing network, such as one having any number ofclients and/or servers. Referring now to FIG. 15, there is shown a blockdiagram depicting an exemplary architecture 30 for implementing at leasta portion of a system according to an embodiment of the invention on adistributed computing network. According to the embodiment, any numberof clients 33 may be provided. Each client 33 may run software forimplementing client-side portions of the present invention; clients maycomprise a system 20 such as that illustrated in FIG. 14. In addition,any number of servers 32 may be provided for handling requests receivedfrom one or more clients 33. Clients 33 and servers 32 may communicatewith one another via one or more electronic networks 31, which may be invarious embodiments any of the Internet, a wide area network, a mobiletelephony network (such as CDMA or GSM cellular networks), a wirelessnetwork (such as WiFi, WiMAX, LTE, and so forth), or a local areanetwork (or indeed any network topology known in the art; the inventiondoes not prefer any one network topology over any other). Networks 31may be implemented using any known network protocols, including forexample wired and/or wireless protocols.

In addition, in some embodiments, servers 32 may call external services37 when needed to obtain additional information, or to refer toadditional data concerning a particular call. Communications withexternal services 37 may take place, for example, via one or morenetworks 31. In various embodiments, external services 37 may compriseweb-enabled services or functionality related to or installed on thehardware device itself. For example, in an embodiment where clientapplications 24 are implemented on a smartphone or other electronicdevice, client applications 24 may obtain information stored in a serversystem 32 in the cloud or on an external service 37 deployed on one ormore of a particular enterprise's or user's premises.

In some embodiments of the invention, clients 33 or servers 32 (or both)may make use of one or more specialized services or appliances that maybe deployed locally or remotely across one or more networks 31. Forexample, one or more databases 34 may be used or referred to by one ormore embodiments of the invention. It should be understood by one havingordinary skill in the art that databases 34 may be arranged in a widevariety of architectures and using a wide variety of data access andmanipulation means. For example, in various embodiments one or moredatabases 34 may comprise a relational database system using astructured query language (SQL), while others may comprise analternative data storage technology such as those referred to in the artas “NoSQL” (for example, HADOOP CASSANDRA™, GOOGLE BIGTABLE™, and soforth). In some embodiments, variant database architectures such ascolumn-oriented databases, in-memory databases, clustered databases,distributed databases, or even flat file data repositories may be usedaccording to the invention. It will be appreciated by one havingordinary skill in the art that any combination of known or futuredatabase technologies may be used as appropriate, unless a specificdatabase technology or a specific arrangement of components is specifiedfor a particular embodiment herein. Moreover, it should be appreciatedthat the term “database” as used herein may refer to a physical databasemachine, a cluster of machines acting as a single database system, or alogical database within an overall database management system. Unless aspecific meaning is specified for a given use of the term “database”, itshould be construed to mean any of these senses of the word, all ofwhich are understood as a plain meaning of the term “database” by thosehaving ordinary skill in the art.

Similarly, most embodiments of the invention may make use of one or moresecurity systems 36 and configuration systems 35. Security andconfiguration management are common information technology (IT) and webfunctions, and some amount of each are generally associated with any ITor web systems. It should be understood by one having ordinary skill inthe art that any configuration or security subsystems known in the artnow or in the future may be used in conjunction with embodiments of theinvention without limitation, unless a specific security 36 orconfiguration system 35 or approach is specifically required by thedescription of any specific embodiment.

FIG. 16 shows an exemplary overview of a computer system 40 as may beused in any of the various locations throughout the system. It isexemplary of any computer that may execute code to process data. Variousmodifications and changes may be made to computer system 40 withoutdeparting from the broader scope of the system and method disclosedherein. Central processor unit (CPU) 41 is connected to bus 42, to whichbus is also connected memory 43, nonvolatile memory 44, display 47,input/output (I/O) unit 48, and network interface card (NIC) 53. I/Ounit 48 may, typically, be connected to keyboard 49, pointing device 50,hard disk 52, and real-time clock 51. NIC 53 connects to network 54,which may be the Internet or a local network, which local network may ormay not have connections to the Internet. Also shown as part of system40 is power supply unit 45 connected, in this example, to a mainalternating current (AC) supply 46. Not shown are batteries that couldbe present, and many other devices and modifications that are well knownbut are not applicable to the specific novel functions of the currentsystem and method disclosed herein. It should be appreciated that someor all components illustrated may be combined, such as in variousintegrated applications, for example Qualcomm or Samsungsystem-on-a-chip (SOC) devices, or whenever it may be appropriate tocombine multiple capabilities or functions into a single hardware device(for instance, in mobile devices such as smartphones, video gameconsoles, in-vehicle computer systems such as navigation or multimediasystems in automobiles, or other integrated hardware devices).

In various embodiments, functionality for implementing systems ormethods of the present invention may be distributed among any number ofclient and/or server components. For example, various software modulesmay be implemented for performing various functions in connection withthe present invention, and such modules may be variously implemented torun on server and/or client components.

The skilled person will be aware of a range of possible modifications ofthe various embodiments described above. Accordingly, the presentinvention is defined by the claims and their equivalents.

What is claimed is:
 1. A virtual reality and mixed reality enhancedexercise machine, comprising: an exercise machine; and a body joystickcomprising: at least one rigid, semi-rigid, or flexible support armaffixed to the exercise machine, another object, or a fixed surface; atleast one attachment device affixed to the at least one support arm, andconfigured to be attached to a portion of a user's body; at least onesensor attached to, or integrated into, the body joystick, andconfigured to detect movement of at least a portion of a user's bodyduring use; and at least one haptic feedback device attached to, orintegrated into, the body joystick, and configured to provide sensoryfeedback to the user during use; and a control system comprising atleast a plurality of programming instructions stored in the memory of,and operating on at least one processor of, a computer system, whereinthe plurality of programming instructions, when operating on theprocessor, cause the computer system to: collect input data based onposition and motion of a user's body from the body joystick; transmitthe body joystick input data to a virtual reality or mixed realitymachine or software application; receive data back from the virtualreality or mixed reality machine or software application based on theuser's interactions within the virtual reality or mixed realityenvironment; and direct the operation of the at least one hapticfeedback device based on the received data to indicate some interactionwithin the virtual reality or mixed reality environment.
 2. The systemof claim 1, wherein the attachment device comprises a strap, belt, vest,waist harness, or other device wearable on the human body.
 3. The systemof claim 1, wherein the haptic feedback comprises restriction of theuser's movement.
 4. The system of claim 1, wherein the entire exercisemachine is mounted on a motorized, rotatable platform, which rotates theentire exercise machine in the direction indicated by the motion of allor some portion of the user's body.
 5. A method for operating a virtualreality and mixed reality enhanced exercise machine, comprising thesteps of: measuring the movement and position of a user while exercisingusing a body joystick attached to the user's body; transmitting the bodyjoystick movement and position data to a virtual reality or mixedreality machine or software application; receiving data back from thevirtual reality or mixed reality machine or software application basedon the user's interactions within the virtual reality or mixed realityenvironment; and directing the operation of the at least one hapticfeedback device based on the received data to indicate some interactionwithin the virtual reality or mixed reality environment.
 6. The methodof claim 5, comprising the additional step of attaching the joystick tothe user's body using a strap, belt, vest, waist harness, or otherdevice wearable on the human body.
 7. The method of claim 5, comprisingthe additional step of using haptic feedback to restrict the user'smovement.
 8. The system of claim 5, comprising the additional step of:mounting the entire exercise machine on a motorized, rotatable platformwhich rotates the entire exercise machine in the direction indicated bythe motion of all or some portion of the user's body.